Effective neural network training with adaptive learning rate based on training loss
نویسندگان
چکیده
منابع مشابه
Effective Neural Network Training with a Different Learning Rate for Each Weight
Batch training algorithms with a di erent learning rate for each weight are investigated. The adaptive learning rate algorithms of this class that apply inexact one{ dimensional subminimization are analyzed and their global convergence is studied. Simulations are conducted to evaluate the convergence behavior of two training algorithms of this class and to compare them with several popular trai...
متن کاملEffective Neural Network Training with a Different Learning Rate for Each Weight
l3al,cli training algoritlims with it different learning r a k Cor each wciglit are invcstigatcd. The ada,ptive learnirig rate algorithiiis or t.liis class tllat apply iuexact o i i e dimcnsioual s i ~ l ~ ~ i i i n i m i ~ a t i o ~ i arc n ialyxd and their glohid coiivcrgciice is studied. Simirlatioiis tire coiiducl,ed to evaluat,e l.hc coiivergciicc behavior of two triiiiiing algorii.1ims oC...
متن کاملAn Adaptive Learning Rate for Training Ring-Structured Recurrent Network
A new adaptive learning rate is proposed based on the Lya-punov stability theory for training the Ring-Structured Recurrent Network (RSRN). The adaptive rate is a suucient condition to guarantee the stability and the most rapid convergence of the RSRN dynamic backpropagation algorithm, and it is easily determined in a direct and non-trial manner. Examples of training the RSRN to predict time se...
متن کاملSliding Mode Control Approach for Training On-line Neural Networks with Adaptive Learning Rate
This chapter includes contributions to the theory of on-line training of artificial neural networks (ANN), considering the multilayer perceptrons (MLP) topology. By on-line training, we mean that the learning process is conducted while the signal processing is being executed by the system, i.e., the neural network continuously adjusts its free parameters from the variations in the incident sign...
متن کاملTwo Novel Learning Algorithms for CMAC Neural Network Based on Changeable Learning Rate
Cerebellar Model Articulation Controller Neural Network is a computational model of cerebellum which acts as a lookup table. The advantages of CMAC are fast learning convergence, and capability of mapping nonlinear functions due to its local generalization of weight updating, single structure and easy processing. In the training phase, the disadvantage of some CMAC models is unstable phenomenon...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
ژورنال
عنوان ژورنال: Neural Networks
سال: 2018
ISSN: 0893-6080
DOI: 10.1016/j.neunet.2018.01.016